37 research outputs found
Spiking memristor logic gates are a type of time-variant perceptron
Memristors are low-power memory-holding resistors thought to be useful for
neuromophic computing, which can compute via spike-interactions mediated
through the device's short-term memory. Using interacting spikes, it is
possible to build an AND gate that computes OR at the same time, similarly a
full adder can be built that computes the arithmetical sum of its inputs. Here
we show how these gates can be understood by modelling the memristors as a
novel type of perceptron: one which is sensitive to input order. The
memristor's memory can change the input weights for later inputs, and thus the
memristor gates cannot be accurately described by a single perceptron,
requiring either a network of time-invarient perceptrons or a complex
time-varying self-reprogrammable perceptron. This work demonstrates the high
functionality of memristor logic gates, and also that the addition of
theasholding could enable the creation of a standard perceptron in hardware,
which may have use in building neural net chips.Comment: 8 pages, 3 figures. Poster presentation at a conferenc
My, and others', spiking memristors are true memristors: a response to R.S. Williams' question at the New Memory Paradigms: Memristive Phenomena and Neuromorphic Applications Faraday Discussion
At the Faraday Discussion, in the paper titled `Neuromorphic computation with
spiking memristors: habituation, experimental instantiation of logic gates and
a novel sequence-sensitive perceptron model' it was demonstrated that a large
amount of computation could be done in a sequential way using memristor current
spikes (d.c. response). As these spikes are found in many memristors (possibly
all), this novel approach could be highly useful for fast and reproducible
memristor circuits. However, questions were raised as to whether these spikes
were actually due to memristance or merely capacitance in the circuit. In this
longer version of the Faraday Discussion response, as much information as is
available from both published and unpublished data from my lab is marshalled
together. We find that the devices are likely imperfect memristors with some
capacitance, and that the spikes are related to the frequency effect seen in
memristor hysteresis curves, thus are an integral part of memristance.Comment: Long form of a Faraday Discussions commen
When and where do feed-forward neural networks learn localist representations?
According to parallel distributed processing (PDP) theory in psychology,
neural networks (NN) learn distributed rather than interpretable localist
representations. This view has been held so strongly that few researchers have
analysed single units to determine if this assumption is correct. However,
recent results from psychology, neuroscience and computer science have shown
the occasional existence of local codes emerging in artificial and biological
neural networks. In this paper, we undertake the first systematic survey of
when local codes emerge in a feed-forward neural network, using generated input
and output data with known qualities. We find that the number of local codes
that emerge from a NN follows a well-defined distribution across the number of
hidden layer neurons, with a peak determined by the size of input data, number
of examples presented and the sparsity of input data. Using a 1-hot output code
drastically decreases the number of local codes on the hidden layer. The number
of emergent local codes increases with the percentage of dropout applied to the
hidden layer, suggesting that the localist encoding may offer a resilience to
noisy networks. This data suggests that localist coding can emerge from
feed-forward PDP networks and suggests some of the conditions that may lead to
interpretable localist representations in the cortex. The findings highlight
how local codes should not be dismissed out of hand
Are there any ‘object detectors’ in the hidden layers of CNNs trained to identify objects or scenes?
Various methods of measuring unit selectivity have been developed with the
aim of better understanding how neural networks work. But the different
measures provide divergent estimates of selectivity, and this has led to
different conclusions regarding the conditions in which selective object
representations are learned and the functional relevance of these
representations. In an attempt to better characterize object selectivity, we
undertake a comparison of various selectivity measures on a large set of units
in AlexNet, including localist selectivity, precision, class-conditional mean
activity selectivity (CCMAS), network dissection,the human interpretation of
activation maximization (AM) images, and standard signal-detection measures. We
find that the different measures provide different estimates of object
selectivity, with precision and CCMAS measures providing misleadingly high
estimates. Indeed, the most selective units had a poor hit-rate or a high
false-alarm rate (or both) in object classification, making them poor object
detectors. We fail to find any units that are even remotely as selective as the
'grandmother cell' units reported in recurrent neural networks. In order to
generalize these results, we compared selectivity measures on units in VGG-16
and GoogLeNet trained on the ImageNet or Places-365 datasets that have been
described as 'object detectors'. Again, we find poor hit-rates and high
false-alarm rates for object classification. We conclude that signal-detection
measures provide a better assessment of single-unit selectivity compared to
common alternative approaches, and that deep convolutional networks of image
classification do not learn object detectors in their hidden layers.Comment: Published in Vision Research 2020, 19 pages, 8 figure
Interactivity:the missing link between virtual reality technology and drug discovery pipelines
The potential of virtual reality (VR) to contribute to drug design and
development has been recognised for many years. Hardware and software
developments now mean that this potential is beginning to be realised, and VR
methods are being actively used in this sphere. A recent advance is to use VR
not only to visualise and interact with molecular structures, but also to
interact with molecular dynamics simulations of 'on the fly' (interactive
molecular dynamics in VR, IMD-VR), which is useful not only for flexible
docking but also to examine binding processes and conformational changes.
iMD-VR has been shown to be useful for creating complexes of ligands bound to
target proteins, e.g., recently applied to peptide inhibitors of the SARS-CoV-2
main protease. In this review, we use the term 'interactive VR' to refer to
software where interactivity is an inherent part of the user VR experience
e.g., in making structural modifications or interacting with a physically
rigorous molecular dynamics (MD) simulation, as opposed to simply using VR
controllers to rotate and translate the molecule for enhanced visualisation.
Here, we describe these methods and their application to problems relevant to
drug discovery, highlighting the possibilities that they offer in this arena.
We suggest that the ease of viewing and manipulating molecular structures and
dynamics, and the ability to modify structures on the fly (e.g., adding or
deleting atoms) makes modern interactive VR a valuable tool to add to the
armoury of drug development methods.Comment: 19 pages, 3 figure
Subtle Sensing:Detecting Differences in the Flexibility of Virtually Simulated Molecular Objects
During VR demos we have performed over last few years, many participants (in
the absence of any haptic feedback) have commented on their perceived ability
to 'feel' differences between simulated molecular objects. The mechanisms for
such 'feeling' are not entirely clear: observing from outside VR, one can see
that there is nothing physical for participants to 'feel'. Here we outline
exploratory user studies designed to evaluate the extent to which participants
can distinguish quantitative differences in the flexibility of VR-simulated
molecular objects. The results suggest that an individual's capacity to detect
differences in molecular flexibility is enhanced when they can interact with
and manipulate the molecules, as opposed to merely observing the same
interaction. Building on these results, we intend to carry out further studies
investigating humans' ability to sense quantitative properties of VR
simulations without haptic technology
Evolving spiking networks with variable resistive memories
Neuromorphic computing is a brainlike information processing paradigm that requires adaptive learning mechanisms. A spiking neuro-evolutionary system is used for this purpose; plastic resistive memories are implemented as synapses in spiking neural networks. The evolutionary design process exploits parameter self-adaptation and allows the topology and synaptic weights to be evolved for each network in an autonomous manner. Variable resistive memories are the focus of this research; each synapse has its own conductance profile which modifies the plastic behaviour of the device and may be altered during evolution. These variable resistive networks are evaluated on a noisy robotic dynamic-reward scenario against two static resistive memories and a system containing standard connections only. The results indicate that the extra behavioural degrees of freedom available to the networks incorporating variable resistive memories enable them to outperform the comparative synapse types. © 2014 by the Massachusetts Institute of Technology